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Description 

FlFl n OF THE INVENTION 
5 The present invention relates to digital detection for eye color detects of a subject in a digital image. 

BACKGROUND OF THE INVENTION 

In color flash photography a phenomenon know as red eye can occur. The term "red eye" refers to a person's 
io photographically reproduced pupil(«) when they appear red instead of their normal black coloration. [This effect also 
occurs in flash photography of animals, although the characteristic color may not be red.] The reddish colored light i is 
the result of the light produced by the flash reflecting off the eye's layer immediately behind the retina. To prevent this 
problem two procedures have been used during image capture: (1) increase the ambient fight levels or (2) separate 
the flash from the camera by several inches. The first solution reduces the size of the pupil thus reducing the quantity 
is of fight reflected and also lessens the amount of required flash illumination. The second measure prevents the lens 
from deling the reflected light since it is confined to a narrow beam directed back towards its source-the flash. 
Unfortunately the amateur photographer is not always in a position to execute either of the above solutions, in general 
the amateur uses a flash to account for the location's lack of ambient light In addition, many point^and-shoot cameras, 
as well as some SIR'S, have the Hash located closely to the lens. 
20 After the image has been digitally captured it is possible to correct the eye color defects. US-A-5,1 30,789 describes 
a method by which a region's chroma and luma channels are modified after an operator identifies a region surrounding 
an eye, and was used as a basis for the preamble of claim 1 . 



SUMMARY OF THE INVENTION 

26 

It is an object of this invention to automatically determine a defective colored eye in an image. 

It is another object of this invention to improve the correction by providing separate corrections for body, border, 
and glint pixels; and also provide for performing correction at different resolutions than that of detection. 

These and other objects of the present invention are achieved by the characterizing features of daim 1 . Preferred 
so embodiments may be gathered from the dependent claims. 

One pf the preferred methods of the present invention is an automated method of detecting for eye color defects 
of a subject in an image due to flash illumination as set forth in claim i . 

The method used by the present invention maintains the *7G& patent criterion of recoloration without destroying 
the region's luma fluctuation. However, the present invention automatically supplies two of the three inputs necessary 
35 to perform the disclosure of the '78g patent. Only the general region erf interest is required at the beginning of the 
present invention as opposed to the more precise region requirements of the 789 patent Also note that the present 
invention divides each pupil into three pixel categories: (1 ) body pixels, (2) border pixels, and (3) glint pixels. Each of 
these pixel types is treated differently in order to create a more natural appearing. correction. Furthermore the image 
resolution from which the region of interest was obtained does not necessarily need to be the resolution at which the 
40 present invention corrects. 



ADVANTAGES 

The present invention is effective to detect eye color defects due to flash illumination and has the advantages of: 

45 

(1) minimizing operator intervention, 

(2) achieving more accurate eye color defect corrections than was heretofore possible, 

(3) allowing for eye color defects to be detected at one image resolution while the correction process takes place 
in another image resolution, and 

so (4) sub-dividing the eye color defect pixels into three categories (body, border, and glint) in order to render a more 

natural appearing eye. 



BRIEF DESCRIPTION OF THE DRAWINGS 

55 Figure 1 illustrates a system in which the present invention is embodied; 

Figure 2 illustrates a top level flow diagram Of certain aspects of the present invention; 
Figure 3 illustrates a detailed flow diagram of the detection phase of Figure 2; 
Figure 4 illustrates a detailed flow diagram of the fix phase of Figure 2; 



2 
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Figure 5 illustrates the hue. lightness, and saturation space which is used in accordance with the present inv ^ on j 
Figures 6a-6c illustrate the Hue, Lightness. Saturation trapezoidal scoring usee* in the First Scoring stage 230 of 
Figure 3 and in the Second Scoring stage 250 shown in Figure 3; 
Figure 7 illustrates Area trapezoidal scoring used in the present invention; 
5 Figures Ba and 8b illustrate 45* trape2oidal scoring used in the present invention; 

Figure 9 illustrates an example of the scaling up method shown in the Correcting stage 380 shown in Figure 4, 
Figure 10 illustrates an example of the scaling down method outlined in the Correcting stage 380 of Figure 4; and 
Figure H illustrates an example eye odor defect whereby each pixel of the eye color defect is deemed as either 
a: body pixel, border pixel, or glint pixel 

10 

DETAILED DESCRIPTION 

Referring to Figure 1 , the present invention is implemented using a digitizing scanner 1 0. a control and logic device 
24 such as a computer with a display device 26. and an optical disc writer 36. As shown, a strip of film 1 5, containing 

15 one or more frames 18 of developed film, is placed into a scan gate 12 where the strip of film 15 is scanned under 
direction of control and logic circuitry 30. As each frame 18 is scanned the resultant image data represented by block 
22, is digitized and transmitted to a memory 28 for storage. The computer 24 process the stored data, in a manner to 
be described, to provide output image data 32 which may be written to an optical disc 34 by the optical disc writer 36 
to provide a report as to the characteristics of the anomalies. The digitized scanner 10 is capable of quantizing pixel 

20 values into multiple brightness levels in separate red. green, and blue channels, A minimum number of brightness 
levels would be approximately 64 for an adequate quality Image with a typteal number being 256. Although tne vnage 
data 32 has bsen described as originating from the operation of scanning film, it is well understood that other well 
known techniques may be used to provide the image data 

Tne computer 24 operates the present invention. Referring now to Figure 2. this invention can be broken down 

2S into two distinct phases: (1) the detection phase 200 and (2) the fix phase 300. Thedetection phase 200 can be broten 
down into five distinct stages (see Figure 3): (1 ) identification oi eye color defect candidate pixels 210, [2y segmentation 
220, (3) first scoring 230, (4) region growth (or pixel score transition identification) 240. and (5) second sconng 250. 
Each stage of the detection phase 200 accesses the results produced by the previous stage creating a data processing 
chain Tne fix phase 300 can be broken down into two sections (see Figure 4): (1 ) scaling 370 and (2) correcting 380. 

30 The resolution section can in turn be broken down into three stages: (1 ) lower resolution 31 0. (2) same resolution 320, 
and (3) higher resolution 330. The correction section can also be broken down into three stages: (1) glint correction 
340. (2) body correction 350, and (3) border correction 350. 

It is important to note the necessity of this data processing chain. In the case of red ey© defects if one were simply 
to assume that all reddish pixels in an image were defects, then numerous non-red eye pixels would be included with 

ss the set of true red eye pixels. Red eye is comprised of both color and spatial cues. To depend on one, but not the other 
set of cues, will produce incorrect results. It is the building up of clues, produced by each stage in the data processing 
chain, which allows the present invention to discern between red eye and non-red eye pixels. 

What makes red eye so noticeable and distracting to a viewer is the concentration of red elements (e.g.. pixels on 
a CRT) surrounded by non-red elements. An elements' coloration is determined by fts combination of the red. green, 

40 and blue characterizing values. Therefore, one couJd conclude that the elements which comprise a red eye region 
contain more red than green or blue. In fact, experimental results show red eye pixels to observe, on average, a 1.7: 
i red to maximum (green, blue) intensity ratio in the video gamma metric. By increasincyfclecreasing this ratio the 
invention decreases/increases its sensitivity to red. 

By converting an element's RGB values to HLS (hue. Tightness, and saturation) space, see Figure 6. one can 

45 derive the kind of red observed in flash illuminated human eyes. The following derh/ed values are those used in an 
existing implementation, but may be adjusted to modify the sensitivity of the present invention: 



so 



700 5 Hue £1010 
40£Luma£ 166 



65 £ Saturation £ 256 



Therefore the ratio and the MimensionaJ HLS region comprise the criteria used In the first stage of the present 
invention. Any pixel in the given image which adheres to these criteria is represented as a one in a pre-zerocd out bit 
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map the color bit map. A bit map is a section of memory which holds ones or zeros. This binary data can repres e ni 
many things depending on the program which interprets it. In the present invention, the bit maps represent pixels Which 
are eye color defect pixel candidates. Since these maps use only a single bit to represent a piece of data, this data 
structure can save vast amounts of physical memory. Thus, at the end of this stage only those pixels which pass the 

5 above criteria are represented in the color bit map. For a detailed discussion of hue, lightness, and saturation space 
see Computer Graphics: A Quarterly Report of SIGGRAPH-ACM, Vol. ia No. 3. Aug. 1979. pp. Ill - 37 & 38. 

Up to this stage the present invention has been working on individual pixels; eliminating those wtuch did not meet 
the Identification of Eye Color Defect Candidate Pixels stage 210 criteria in Figure 3. It is now time to group these 
pixels into spatially contiguous groups, termed segments. 

io The process begins by raster scanning the color bit map until a non*eroed bit is encountered This bit is the seed 
bit Once found, the seed bit is set to one in the pre-zeroed marked bit map. Next, the segmentor enters a loop which 
terminates when no bits are -on" in the marked bit map. The first step of the loop is to zero out the current bit in the 
color bit map. Next, a function is called which searches (in Q-connected fashion) around the current bit in the color bit 
map for "on" bits. As *on' bits are found, they are placed into the marKed bit map if and only if they are not also -on" 

is in the seed bit map. Once all S-ccnnected "on- bits have been found, the function returns to the segmentor loop. The 
next step of the loop is to zero out the current bit in the marked bit map and turn -on* the current bit m the seed bit 
map Finally, a function is called which searches the marked bit map for the top left non-zeroed bit This bit now becomes 
the current bit and the loop cycles again. Eventually the segmentor loop terminates. At this point each segment is 
validated based on area and shape. Specifically, there are two criteria, eccentricity and number of pixels. The eccen- 

20 tricrty of a segment must be greater than a minimum eccentricity and less than a maximum eccentricity, where 0.0 is 
perfectly round and 1 .0 is maximally elongated (i.e., a line). This range of eccentricity values eliminates linear segment 
types (i.e., pupils are roundish). See Appendix A for the current equations used to implement an eccentricity value. A 
segment must contain a predetermined minimum and maximum area (in pixels). A segment wfth fewer than the mini- 
mum number of pixels is generany indistinguishable from other image objects (e.g., a patch of skin). If validated, then 

ss this segment is entered into the classmap bit map. The zeroing out of the seed bit map ends this part of the segmentor. 
Before continuing, it is crucial that the reader understand that only two of the four bit maps, the color and classmap bit 
maps, could currently contain "on" bits. The color bit map might contain V>n" bits which belong to other segments, and 
the classmap bit map can only contain those segments previously found and the current segment. 

Lastfy, the raster scan of the color bit map continues. When the next non-zeroed bit in the color bit map is encoun- 

30 tered it is designated as the seed bit and the process begins anew. This continues until there are no 'on' bits in the 
color bit map. 

As a segment is classified, two tasks are performed: (1) insertion of the new segment into the classmap bit map 
and (2) compilation of data on the segment is accumulated. This data includes eccentricity, a measure of a segment's 
circular nature; the number of pixels contained within the segment (i.e., area); the segment's score; the best pixel score 
ss (i.e., the largest pixel score occurring in the segment); the best peters coordinates; and the coordinates of the segments 
cemW Each segment found has its corresponding statistics saved so that the following stages may access them. 

Scoring; (see Figure 3) 

40 A continuous-weighting approach to scoring a pixel is taken in the present invention, various crfterta (hue, lightness, 
saturation, and the ratio test) are taken into consideration and based upon them an eye color defect score is given. 
Scores range from zero to one, inclusive. 

In the present invention a pixel is scored based on its HLS values and whether or not the pixel passes the ratio 
Criterion (i.e., a bi-value score). Each pixel in an image contains a hue. lightness, and saturation value. Given these 
4$ values a score for each of these channels can be calculated (sc©re hy6 , score j(gh[noS3 , and score^raiion)' These scores 
are produced based on the information presented in Figures 6a-6c. A score for a given channel vafue equals the 
corresponding trapezoidal weighting function for that value. Next read the score off the y-axia. For example, if a grven 
pixel has the following HLS values: 
H»73, 
so L = 75, 

S = 63. then the corresponding scores would be: 
«core hgP = 0.5 

SCO r ©fighh«e= 1 - 0 

3*0^^^ = 0.032. 
SS In pseudo-code, a score is calculated by: 
if (valuep^value^then 

score = 0.0 
else if (value p > valued then ■ 
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score * o.o 
else if(valuG, s value p < value 2 ) then 

score = (value,, - values J/rvaiue^ " valued 
else if(valus 3 value p <, < value 4 ) then 
5 score = (value., - vaJue 4 )/(value3 - vaJue^; 

where valuep is the channel (hue, tightness, or saturation) of the pixel in question, value, represents the first 
point of the trapezoid (hi. li,orsl). value 2 represents the second point (h2, 12, ors2). valueg represents the third point 
(h3, 13, or s3), and value 4 represents the fourth point (h4, 14, or s4). Lastly, these three scores are multiplied together 
producing a final pixel score ranging from O.O to 1 .0, inclusive. 
70 The purpose for multiplying individual channel scores together is so that if a pixel fails a channel test (falls outside 
the trapezoidal region), then that pixel will receive an overall score of zero (La., not a eye oolor defect candidate). Also, 
a very large score In one channel can partially compensate' for low-but acceptable-scores in the other channels. 
A segment's score is determined by calculating the average score of the pixels contained within it. 
The Region Growth stage 240 is a process for cleaning up segments by eliminating questionable sub-regions 
is which passed the color threshold tests bat appear to have unlikely spatial variations. 

Digital imaging region growth is a method by which pixels are annexed to a region's already existing pixels. This 
is analogous to the growth of crystals. For a crystal to begin growing, a piece of dust (or some other material) must 
exist. Then, as a solution Is introduced, some of the dissolved particles precipitate out of solution and begin forming 
the crystal structure. Growth ends when the particles in the solution drop below a certain concentration. In a similar 
so way. the growth of a digital region begins with a supplied seed pixel and ends when the connected pixels no longer 
meet the defined criteria of similarity. A segment's seed pixel Is obtained during the Segmentation stage 220. This pixel 
is designated as the coordinates of the best pixel in a given segment 

To start the Region Growth stage 240 loops through each segment found in the Segmentation stage 220. As 
described above, each segment contains a best pixel. This pixel is the seed pixel. The method searches (in an 9-con- 
25 nected pattern) around the current pixel and locates all pixels which have scores that are: 

$COre oUJrtftl pIxaI - 0.5 £ SCOrOrwighbOrfrtS pixel 

scorB neighbcring pixel - ^O^cumeni segment L 

Those neighboring pixels which pass one or both of the above two criteria are annexed to the growing regrcn. only 
those pixels which were part of the segment, after the Segmentation stage 220, are considered during this stage. This 
so process of annexing pixels continues for each segment una no old segment pixels are left to be verified. 

Since the resultant segments from this stage may differ from those of the Segmentation stage 220 (i.e., pixels may 
have been dropped from the segment's make up), new segment statistics and scores are updated 250. 

At this point there is nothing more to do than assume that the segment with the best score Is an eye color defect, 
termed the primary eye (or primary eye segment). If more than one eye is expected in the image window, then an 
ss additional search is conducted for other eye color defects, tenmed secondary eyes (or secondary eye segments). Based 
on the primary eye segmenfs area (or number of pixels) and the 45° criteria a new score is calculated for the other 
defective eye segment candidates. [The 45* criteria mandates that the secondary eye must be located somewhere 
within ± 45* of the horizontal, which runs through the centroid of the primary eye (see Figures Ga and 8b). Any segments 
which are not in this area are given scores o! zero (i.e.. are not considered to be secondary eye candidates).] The area 
«> score (scored is determined based on the triangular weighting function presented in Figure 7. As with the previous 
scoring methods, the area scores range from 0.0 to 1.0, inclusive. This score^ and the score^ are multiplied to the 
score of the segment in question, in order to arrive at the segmenfs new score. The segment with the top remaining 
score is assumed to be the secondary eye. 

Avery important feature of the present invention centers around the fix phase 500 and its flexibility. By this phase 
45 the classmap bit map should contain the eye color defects within the image. Now. the invention must replace each eye 
color defect with a natural appearing pupil. To accomplish this rendering two important issues must be investigated: 
(1) scaling 370. and (2) correcting 380. 



so 



Scaling: (see Figure 4, 370). 

The flexibility alluded to above involves the interdependences of the detection phase 200 with the fbc phase 300. 
One might wish to perform the detection phase 200 in some image resolution, but perform the fix phase 300 in another. 
To account for this flexibility, the Scaling section 370, in Figure 4, is subdivided into three sections: (1) fixing at the 
same resolution 320 as detection, (2) fixing at a higher resolution 330, and (3) fixing at a lower resolution 310, Each 
of theso sections is handled differently. To fix at the same resolution 320 requires no added classmap bit map handling. 
All that is needed is to correct each pixel which is represented in the classmap bit map (see Correcting below). To fix 
at higher or lower resolutions requires the creation of a new bit map, the scaledCtassMap. 

The problem when correcting at a higher resolution 330. is that each pixel in the classmap actually has contributions 
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from several pixels at the higher resolution. Some of these pixels may be ©ye color defect pixels, but others may not. 
Thus, a refined eye color defect selection process must be performed at the higher resolution. 

The process of fixing an Image at a higher resolution entails the formation of a scaled up version of the classmap 
bit map The first step of the process is to pass the scaled version of the original image (the image which the detection 

s phase 200 worked on) to the Identification of Eye Color Defect Candidate Pixels stage 210. This stage returns a scaled- 
Color bit map. Next, each segment contained within the classmap bit map is examined. As each segment is encountered 
Ms best pixel's coordinates are scaled up by the appropriate x-/y-ecale factors. In theory these coordinates will be at 
most ± the xVy-scale factor number of coordinates away from the original coordinates multiplied by their scale factor 
value, respectively. For examp le, if an image contained one eye color defect and the present invention correctly detected 

io ft, then the classmap bit map shown in the upper left-hand comer of Figure 9 would be the result. Furthermore, if the 
user requested that the fix phase 300 be performed on the image scaled by 2 (in both the x and y directions), then the 
scaledColor bit map shown in the lower-right comer of Figure 9 would be the result. Figure 9 depicts the bit located at 
(4 2) in the classmap has been designated as the best pixel (see Segmentation 220). When these coordinates are 
multiplied by the respective scale factors (two for both coordinates in this example), the resulting bit is located at £8, 

js 4)-designated in Figure 9 by a 'X'. Unfortunately this bit location is not classified as a eye color defect candidate in 
the scaledColor bit map. Therefore, a search is undertaken to find the closest eye color defect candidate in the scaled- 
Color bit map- Once found, this location is used as the seed point for the Segmentation stage 220. Finally the Region 
Growth stage 240 is invoked and a valid class map bit map, the seal edClass Map, is produced. All that is left now is to 
correct the pixels in the scaled version of the original image which are categorized as eye color defect pixels in the 

a> ecaledCIassMap bit map (see Correction below). The above outlined methodology does not simply scale up the class- 
map bit map. If one were to replace every bit in the lower resolution with N (where N equals x-scale factor time y-scale 
factor) number of bits, then the resulting class map. the scaledClassMap, would over-estimate the true number of pixels 
contained within the eye color defect. 

A correction at a lower resolution 310 is opposite to a correction in a higher resolution 330 (i.e., each pfcel in the 

as lower resolution image contains information based on multiple pixels from the detect phase 200 image). Some of these 
detect phase 200 image pixels may be classified as eye cotor defect pixels while others may be classified as non-eye 
color delect pixels. The correction process here is substantially simpler than for up-*caling since no region growth 
refinement is needed to produce a more accurate class map. 

For every NxM (where H equals the x-scale factor and M equals the y-scale factor) grouping of pixels in the original 

30 detection phase 1 90 image, there corresponds aefngle lower resolution pixel. Since each pixel inthe NxM pixel grouping 
is not guaranteed to be a eye color defect pixef, the corresponding lower resolution pixel may not possess the necessary 
characteristics to be termed a eye color defect candidate pixel by the detection phase 200. This is a probtem. For a 
program to correct only the parts of a lower resolution pixel which are derived from true eye color defect candidates 
of the original resolution image, correct each lower resolution pixel by a percentage of the full correction. This percent- 

35 age can be derived by knowing the number of eye color defect candidates contained within each NxM grouping of . 
original image pixels. Figure 10 shows a classmap bit map and Hs corresponding scaledClassMap bit map. Again, to 
simply scale each bit in the classmap by the corresponding x-/y-ecale factors to arrive at thB scaledClassMap would 
over-estimate the number of pixels contained within an eye color defect. Furthermore, these pixels would be over 
corrected 

40 

Correcting: (see Figure 4, 380). 

Pupils which exhibit the red eye phenomenon differ from normal pupils in that they are reddish in hue (rather than 
neutral), and are brighter. Correction is generally accomplished by removing a pixers chromaticity and darkening its 
45 luminance. To perform this task, a given pixePs gamma RGB values are converted to YCC values. [YCC space is a 
simple translation of RGB space. To transform an image in RGB space to YCC space, multiply each pixel by the 
following matrix: 



SO 



[RGB]x 



0-229 0.587 0.1 14* 
0.701 -0.587 -0.H4 
-0.229 -0.587 0.8H6 



Y 

c 2 



where R, G, and B represent the gamma-metric red, green, and blue coordinates of a given pixel, and Y, C1. and C2, 
represent the luminance, chroma-channel 1 , and chroma-channel Z coordinates of the pixel in YCC space. The gamma 
RGB and the YCC color metrics are described in the KODAK PhotoCD Products: A Planning Guide for Developers. 



e 



PAGE 17/73 * RCVD AT 3/22/2006 1:18:29 PM [Eastern Standard Time] * SVR: USPT0-EFJ01F-1 / 1 9 ' DNiS:2?38300 * CSID:6508575487 * DURATION (mm-ss):28-10 



MAR. 22. 2006 10:08AM 



HP LEGAL 



NO. 294 P. 18 



BP 0 635 972 B1 

Eastman Kodak Company, 1 992 {part no. DC!200R).l Next, both chroma channels are zeroed out (i.e.. only the pixers 
neutral va | ue \ s \ e ^y At this point, tha luminance channel is multiplied by a factor of 0.35 in order to reducelhe lightness 
ol the pixel's neutral value. (In general, a red eye has an unusually high luminance value due to the over abundance 
Of light produced by the flash of the camera which illuminates the retina. Thus, upon correction, a reduction of this high 

s luminance value is required. This factor, 0.35, is based upon experimental evidence and other values may result in 
natural renderings.) Finally, the YCC values are converted back to gamma RGB values, and set as the pixel's values. 
The above procedure describee Body correction 350. This r* the method by which body pixels-pixels which are 3-con- 
nected only to eye color defect pixels-are desaturated (see Figure 11). Border correction 360 is performed on border 
pixels, pixels which are 3-connected to at least one non-eye color defect pixel and to at least one eye color defect 

10 pixel. Such pixels are only darkened by a 0.1 5 factor. Furthermore, those pixels which are considered to be associated 
with the glints of a eye color defect are sometimes found to contain subtle red hues. Those pixels which: (1 ) are not 
eye color defect pixels. (2) are within 40% of the puplPs calculated radius from the pupil's centroid, and (3) have hue 
values which range from 600 and 1100 f inclusive, are termed glint pixels. To perform Glint correction 340, the chroma 
channels ot glint pixels are zeroed out, while their luminance is left unchanged. 

is This invention has been described with reterence to correcting 'red eye" defects, but those skilled in the art will 

recognize that it is also applicable for conecting for eye color defects in animals, but that different detection and cor- 
rection values specific to the subject's detective color should be used. Moreover, although two specific color spaces 
have been used, others will suggest themselves to those skilled in the art. 
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first scoring stage 




240 


region growth stage 




250 


second scoring 


40 


300 


fix phase 




310 


tower resolution 




320 


same resolution 




330 


higher resolution 




340 


glint correction 


49 


350 


body correction 




360 


border correction 




570 


scaling logic 




380 


correcting logic 



so 

Claims 

1. An automated method of detecting for eye color defects of a subject in a digital image due to flash illumination, 
including 

55 

defining (1 00) a spatial region within the digital image in Which one or more eye color defects may exist, which 
region includes at least a portion of the subject's head; 

sampling pixels within such spatial region for their color content and comparing each such sampled pixel with 
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a plurality of threshold values which are representative of eye color defects to identify possible eye color defect 
pixels (2i0); and 

segmenting (220) the identified possible eye color defect pixels intoone or more spatially contiguous groups; 
characterized by: 

£ a) calculating (230) a first score tor each pixel of each segmented group and for each group based upon a 

pluialrty of features including group size, group shape, coloration, and brightness to identify eye color defect 
group candidates; 

b) selecting a seed pixel based on its score from each identified eye color defect group candidate and deter- 
mining all the neighboring pixels which are within a predetermined score range of their neighboring pixels and 

10 those pixels in a group which represent a significant pixel score transition so that the determined transitions 

tdentify the outer boundary of an eye color defect group candidate (240); 

c) calculating (250) a second score for each pixelfor each eye color defect group candidate based on a plurality 
of f ealures including group size, group shape, coloration, and brightness ; 

d) selecting the best group score of an ey* color defect group candidate and comparing it relative to a prede- 
15 termified threshold group score to determine whether a first eye color defect group is present and identifying 

it as corresponding to the most likely eye; and 

e) determining whether a second actuaJ eye color defect group is present based on whether the area of one 
of the color defect candidate's e^e is within a predetermined threshold ratio of the area of the first actual eye 
color defect group corresponding to the most fikely eye, and whether this group candidate is within a prede- 

20 termined threshold angular subtense of the most likely eye along the predetermined horizontal axis of the 

subjects head. 

2. The method of claim 1 wherein the group size feature is area, the group shapefeature is eccentricity, the coloration 
and brightness features are hue, saturation, lightness, and red to maximum (green, blue) ratio. 



25 



The method of claim 1 or 2 wherein the scores per pixel in steps a) and c) are equal io the product of the scores 
of individual features, and the scores of individual features are weighted functions of the feature values of the pfrefs. 



A. An automated method of detecting and correcting tor eye color defects of a subject in a digital image due to flash 
30 illumination, including the method of any of the preceding claims and further* comprising the steps of: 

f) correcting (300) the defective color in each eye color defect group by: 

(i) determining (370) the correct resolution and whether each pixel at the corrected resolution is an eye color 
defect pixel; 

95 (ii) categorizing (380) the eye color defect pixels at the corrected resolution into either booy border, or glint 

categories; and 

(iii) correcting (340.350.5S0) the eye color defect pixels in the body, border, or glint categories, 

5. The method of claim 4 wherein the threshold value represents minimum and maximum allowed values consistent 
to with the expected color of a eye color defect group. 

6. The method of any of the preceding claims wherein the group score is an average of all the pixels 

7. The method of claim 4 wherein step f) (i) includes determining at a lower resolution a correction factor for each 
45 low resolution pfcei which is a function of the number of eye color defect pixels at the original resolution disposed 

at the same image location. 

a The method of claim 4 wh erein step f) (0 includes categorizing at a higher resolution each original eye color defect 
pixel into eye color defect pixels and non-eye color defect pixels. 

50 

9. The method of claim 4 including desaturating and reducing the illuminance for each body, border, and glint pixel. 



Patentanspruche 

1. Automatisiertes Verfahren zum Er1<finnen von Augenfarbfehrem einer Person In einem Digitalbild, die durch Blitz- 
ifcht verureachtwurden, mttfolgenden Schritten: 
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-< Definieren (100) ernes raumlichen Bereichs innerhalb cfes Digitalbildes, in dem eventuell ein Oder mehrere 
Augenfarbfehler vomanden sind und der zumindest einen Teil des Kopf ss der Person aufweist; 

- Abtasten von Pixeln in diesem Bereich auf ihren Farbanteit und vergleichen. jedes der abgetasteten Pixel mit 
eirrer Viefeahl von Augenfarbfehler kennzeichnenden Schwellenwerten, um nhogliche Augenfarbfehler-Pixel 

5 (210) zu identifizieren; und 

- Segmentieren (220) der identifizierren mdglichen Augsnfarbfehler-Pixel in eine cder mehrere rtumljch zusam- 

menhangende Gruppen; 
gekennzeichnet durch 

10 

a) Berechnen (230) eines ersten Werts fur jedes Pixel jefier Gruppe und fur jede Gruppe auf der Grundlage 
einer Vielzahl von Merkmaten, die GruppengroGe. Gruppenform. Farbung und Heiligkeit umfassen, um Kan- 
didaten fur Augenfarbfehiergruppen zu identifizieren; 

b) Auswahien eines Startpixels auf der G rundlage seiner Bewertung von jedem identifizierten Kandidaten der 
is Augertfarbfehlergnjppen und Bestimmen alier benachbarten Pixel, die innerhalb eines vorgegebenen Bewer- 

tungsberefchs ihrer benachbarten Pixel (iegen, und jener Pixel in einer Gruppe. die einen deutlichen Pixel- 
wertubergang darstellen. so cteB die bestirnmten Gbergange die auBere Grenze elnes Kandidaten (240) der 
Augenfarbfehiergruppen identifizieren; 

c) Berechnen (250) sines zweiten Werts fur jedes Pixel eines jeden Kandidaten der Augenfarbfehiergruppen 
auf der Grundlage ener Vielzahi von Merkmalen. die GruppengroBe, Gruppenform, Farbung und Heiligkeit 
umfassen; 

d) Auswahien der besten Gruppenwertung eines Kandidaten der Augenfarbfehiergruppen und Vergleichen 
dieses Werts mit einem vorgegebenen Gruppenschwellwert, um festzustelien, ob eine erste Augenfarbfehler- 
gruppo vorhanden ist. und Identifizieren dieser Gruppe als eine, die mit hoher WahTscheinlichkeit ein Auge 

25 darsteilt; und 

e) Feststelfen, ob eine zweite tatsacWiche Augenfarbfehlergruppe vortianden ist, darauf basierend, ob der 
Bereich eines der Augen des Farbf ehlerkandidaten innerhalb eines vorgegebenen Grenzwertes des Bereichs 
der ersten tatsachlichen Augenfarbfehlergruppe. die mit hoher Wahrscheinlichkeit ein Auge darsteilt, und ob 
dieser Gruppenkandidat innerhalb eines vorgegebenen Grertzwinkels des wahrscheinlichen Auges entlang 

so der vorgegebenen horizontal Achse des Kopfes der Person llegt. 

2. Verfahren nach Anspruch t . dadurch gekennzeichnet, daB das Merkmal der GruppengroBe Flache, das der Grup- 
penform Exzentrizitat. das der Farbung und Heiligkeit Farbton, Satiigung, Heiligkeit und das Verhaltnis Rot zu 
Maximum (Grun, Blau) bedeutet. 

3. Verfahren nach Anspruch 1 Oder 2, dadurch gekennzeichnet, daB die Werte pro Pixel in Schritt a) und c) gleich 
dem Produkt der Werte der einzetnen Merkmale. und die Werte der errtzelnen Merkmale gewichlete Funktfanen 
der MerkmaJswerte der Pixel sind. 

40 4. Automatisiertes Verfahren zum Erkermen und Korrigieren von Augenfarbfehtern einer Person in einem Digitatbild, 
die durch Blitzlicht verursacht wurden, nach dem Verfahren eines der vomer gehenden Anspruche mit folgenden 
* zusatziichen Schnnen; 
f) Korrigieren (300) des Farbfehlers in jeder Augenfarbfehlergruppe durch: 

45 (j) Bestimmen (570) der richtigen Auflosung und Feststellen, ob jedes Pixel bei der korrigierten Aufl5sung ein 

Augenfarbfehlerpixel 1st; 

(li) Einteilen (380) der Augenfarbfehlerpixel bei korrigierter Auflosung in entweder Korper-. Grenz- oder Blitz- 
reflexionsbereiche; und 

(Hi) Korrigieren (340. 350, 360) der Augenfarbfehlerpixel im Korper-, Grenz- Oder Binzreflexionsbereich. 

5. Verfahren nach Anspruch 4, dadurch gekennzeichnet, daft der Schweltenwert zulassige Mindest- und Hochstwerte 
darsteilt, die der erwarteten Faroe erner Augenfarbfehlergruppe entsprechen. 



so 



6. Verfahren nach einem der vorhergehenden Anspruche, dadurch gekennzeichnet, daft der Gruppenwert ein Durch- 
schnittswert alter Pixel ist 

7. Verfahren nach Anspruch 4, dadurch gekennzeichnet. daB Schritt f) (i) das Bestimmen, bei niedrigerer Auflosung, 
eines Korrekrurfaktors fur jedee Pbcel mit niedriger Auflosung einschiie9t, dor eine Funktion der AnzahJ der Au- 
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genfarbfehlerpixel bender ursprQnglichen Auflosung ist, die an der gleichen Bildstelle vorhanden sind. 

8. Verfahren nach Mspruch 4, oadurch gekennzeichnet. daft Sehrrtt f) <i) das Einteilen. bei hoherer Auf I6su*g, jedes 
ursprungfichen Augenfarbfehlerpixets in Augenfarbfehlerpixel und Nicht-Augenfarbenfehlerpixel einschlieBt. 

5 

9. Verfahren nach Anspruch 4, gekennzeichnet durch Desaturieren und Reduzieren der Beleuchtungsstarke f Or jedes 
Korper-. Grenz- und Blitzreflexionspixef. 

io Ffevendications 

1. Precede automatism de detection des defauts de couleur d'oeil d'un sujet dans une image numerique dus a un 
eclairage au flash, comprenant 

is fa definition (100) d'une region spatiale a rinterieur de rimage numerique dans faquelle un ou plusieurs defauts 

de coufeur d'oeil peuvent exister, (aquelie region comprend au morns une partie de la tete du sujet. 
I'echantillonnage des pixels a Tirtterieur dune telle region Spatiale quant a leur teneur en couleur et la com- 
paraison de chaque tel pixel echantillonne a une pluralite de valeurs de seuil qui sont representatives de 
detains da couIeuT d'oeil afir> ^identifier des pixels de defauts de couleur d'oeil possibles (210) f et 

so ia segmentation (220) des pixels de defauts de couleur d'oeil possibles identifies en un ou piusieurs groupes 

spatialement corrtigus, caractense par 

a) le calcul (230) d'une premiere Evaluation pour chaque pixel de chaque groupe segments et pourchaque 
groupe sur la base d'une pluralite de caracteristiques comprenant la dimension du groupe. la forme du 
groupe , la coloration et la luminosity afm d'identifierdes candidate de groupes de del auts de couleur d'oeil, 

b) la selection dun pixel germe sur la base de son evaluation a partir de chaque candidat de groupe de 
defauts de oouleur d'oeil identffie et la determination de tous tes pixels voisins qui se trouvent a Pinterieur 
d'une plage devaluation predetermines de leurs pixels voisins et de ceux des pixels dans un groupe qui 
represented une transition devaluation de pixels significative* de sorte que les transitions determines* 
jdentifient (a limite externe tfun candidat de groupe de detains de couleur d'oeil (240), 

c) le calcul (250) d'une seconde evaluation pour chaque pixel de chaque candidat de groupe de dSfauts 
de couleur d'oeil sur ia base d'une pluralite de caracteristiques comprenant la dimension du groupe, la 
forme du groups, la coloration et la luminosite, 

d) ka selection de la meilleure evaluation de groupe d'un candidat de groupe de dSfauts de couleur d'oeil 
et la comparaison de celle-ci a une evaluation de groupe de seuil predetermines afin de determiner si un 
premier groupe de defauts de couieu r tfoeil est present, et fidemffication de eel ui-ci comme correspondant 
a Poeil le plus probable, et 

e) la determination de ce qu'un second groupe de defauts de coufeur d'oeil reel est present sur la base 
de ce que raire de I'un des candidats de d£faut de couleur d'oeil est a I'interieur d'un rapport de seuil 
predetermine de Taire du premier groupe de defauts de couleur d'oeil reel correspondant a I'oeil le plus 
probable, et de ce que ce candidat de groupe est a I'interieur d'un angle sous-tendu de seuil predetermine 
de i'oeil (e plus probable suivant t'axe horizontal predetermine de fa tete du sujet 

2, ProcedS scion la revendication 1 , dans lequel la caracteristique de dimension du groupe est i'aire, lacaracteristique 
45 de forme du groups est I'excentricite, les caracteristiques de coloration et de luminosite sont la teirrte, la saturation, 

la luminance, et le rapport du rouge au maximum (vert. bleu). 

3. Precede seton la revendication 1 ou 2, dans lequel lee evaluations par pixel des etapes a) et c) sont egates au 
produit des evaluations des caracteristiques indrviduelles, et les evaluations des caracteristiques irtdividueJles sont 

so , des fonctions ponderees des valeu rs des caracteristiques des pixels. 

4, Procede automatise de detection et de conection des defauts de couleur d'oeil d'un sujet dans une image nurne- 
rique dus a un eclairage au flash, comprenant le precede selon I'une quelconque des revendications precedentes. 
et comprenant en outre les etapes : 

65 f) oorriger (300) la couleur defectueuse dans chaque groupe de defauts de couleur cToeil en : 

(i) determinant (370) la resolution correcte et si chaque pixel a ia resolution corrigee est un pecel de defaut de 
couleur d'oeil, 



so 



55 
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(ii) catSgorisam (3S0) les pixels do d6faut de couleur d'oei! a la resolution corrigee en categories soit de corps, 
sort debordure, sort de reflet, et 

(iii) corrigeant (340, 350, 360) les pixels de defaut de couleur d'oeil dans les categories de corps, de bordure 
ou de reflet 

s 

5. Proced6 salon la revendication 4, dans lequel fa valeur de seuil represent© des valeurs minimum et maximum 
permises coh^rentes avec la couleur previsible d'un 9roupe de delaute de couleur d'oeih 

$. Procecie selon rune quetconque des revendfcations pr£c£dentes. dans requel Evaluation de groupe est une 
io moyenne de tous les pixels. 

7. Proc6d6 selon la revendication 4, dans lequel I'Stape f) (i) comprerri la determination a une resolution inferieuro 
d'un facteur de correction pour chaque pixel a basse resolution qui est une fonction du nombre des pixels de defaut 
de couleur d'oeil a la resolution d'origine disposes au meme emplacement de Pimage. 

6. Proc$d$ selon la revendication 4, dans lequel Petape 1) (i) comprend la categorisation a une resolution superieure 
de chaque pixel de defaut de couleur d'oeil tfongine en des pixels de dtfaut de couleur cPoeil et des pixels qui ne 
sont pas des delauts de couleur d'oeil. 

so 9. precede" selon Fa revendication 4, comprenant la desaturatlon et la reduction de la luminance pour chaque pixel 
de corps, de bofdure et de reflet 
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